J Exp Criminol (2016) 12:1-20 ®@® arsenite 
DOI 10.1007/s11292-016-9252-4 


The continued importance of measuring potentially 
harmful impacts of crime prevention programs: 
the academy of experimental criminology 2014 
Joan McCord lecture 


Anthony A. Braga‘? 


Published online: 17 February 2016 
© Springer Science+Business Media Dordrecht 2016 


Abstract 

Objective Joan McCord was a very influential criminologist and strong advocate for 
measuring potentially harmful effects in well-meaning crime prevention programs. This 
paper demonstrates the continued importance of measuring adverse program effects by 
reviewing the available research evidence on classic and contemporary gang 
streetworker programs. 

Methods This paper draws upon the evaluation findings of existing gang streetworker 
evaluations and presents the unpublished results of a rigorous quasi-experimental 
evaluation of a contemporary gang streetworker program that directly measured wheth- 
er the intervention impacted the violent gun behaviors of treated gangs relative to 
untreated gangs. 

Results Evaluations of classic pre-1970s gang streetworker programs generally found 
that these interventions increased gang delinquency by reinforcing group identity and 
enhancing gang cohesion. Evaluations of contemporary gang streetworker programs 
are mixed, with several studies documenting concerning increases in gang violence. An 
unpublished evaluation found that the streetworker program was well-implemented and 
executed. However, the intervention was associated with increased shootings by and 
against treatment gangs relative to control gangs. 

Conclusion These findings suggest that contemporary gang streetworker programs are at 
high risk of generating unintended adverse outcomes for treated gang members relative 
to their untreated counterparts. Existing and planned programs should be monitored with 
a high degree of vigilance and evaluated with controlled evaluation designs. 
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Introduction 


In the edited volume Fifty Key Thinkers in Criminology, David Farrington (2009:179) 
described Joan McCord as a “brilliant pioneer in criminology” and identified her work 
on the longitudinal Cambridge-Somerville Youth Study randomized experiment as her 
most outstanding contribution to criminological knowledge. As Farrington (2009) 
acknowledged, McCord certainly made many important contributions over the course 
of her career (for a collection of her works, see Sayre-McCord 2007), such as her study 
of the impacts of child-rearing practices on subsequent adult criminal behavior 
(McCord 1979). However, the Cambridge-Somerville Youth Study yielded a variety 
of critical insights into the childhood antecedents of delinquent behavior and, famously, 
was among the first studies to demonstrate unintended harmful effects of a well- 
meaning prevention program (McCord 1978, 1981, 1992). Her subsequent advocacy 
for evaluators to measure potential harmful effects in crime prevention programs 
represents a central aspect of her scholarly legacy. 

Other scholars have expressed concern that too many investigators fail to ask 
whether crime prevention programs had adverse effects and too many research reviews 
lack systematic reporting of iatrogenic outcomes (e.g., see Sherman 2007; Sherman 
et al. 1997; Welsh and Rocque 2014; Werch and Owen 2002). Drawing on her 
experiences and channeling the perspectives of a growing chorus of evidence-based 
crime policy advocates, Joan McCord (2003) made some critical observations on how 
to improve our knowledge on harmful program impacts. She rightly observed that 
publication bias contributed to the lack of attention to adverse outcomes, as scientific 
journals are more likely to publish studies that reported positive treatment effects. To 
minimize the so-called “file-in-the-drawer” problem (Rosenthal 1979), she greatly 
encouraged systematic reviews that searched the so-called “grey literature” for unpub- 
lished studies that might report negative program impacts. 

McCord (2003) also advanced the argument that high-quality designs, with balanced 
comparison groups, were essential in establishing whether a program generated desired 
or undesired impacts on study participants. Without the benefit of a counterfactual, it is 
difficult for program evaluators to identify harmful outcomes. If treated subjects 
experienced adverse effects relative to comparison subjects, the study could immedi- 
ately be halted and unintended harm minimized. Not surprisingly, Joan McCord (2003: 
29) was a strong advocate for increasing the number of randomized controlled trials in 
criminology, making the well-known suggestion, “whenever possible” evaluation 
studies “should employ random assignment.” 

This paper attempts to honor Joan McCord’s rich criminological legacy by 
reaffirming the importance of measuring potentially harmful impacts of crime preven- 
tion programs. The renewed popularity of gang street outreach (commonly called gang 
“streetworker”) programs, despite evaluation studies suggesting that treated subjects 
risk adverse outcomes, underscores the relevance of McCord’s call for continued 
vigilance in this area. The paper begins by briefly reviewing the work of McCord 
and others who found that certain well-meaning individual crime prevention programs 
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generated harmful effects on treated groups of delinquent youth. Theoretical mecha- 
nisms associated with adverse program outcomes, such as construct theory, are then 
discussed. A narrative review subsequently documents that gang streetworker programs 
have also generated unintended harmful impacts by increasing group identity and gang 
cohesion. 

The paper then presents the unpublished findings of a controlled evaluation of a 
Boston gang streetworker program that revealed this well-intentioned program gener- 
ated undesirable increases in gun violence by treated gangs relative to control gangs 
(Hureau et al. 2014). Consistent with prior streetworker program evaluations (Klein 
1971; Wilson and Chermak 2011), the unpublished evaluation suggests that these 
harmful impacts may have been driven by the Boston program’s emphasis on group 
processes already in place that inadvertently increased gang cohesiveness and, in turn, 
gang violence. In effect, the Boston gang streetworker program seemed to reinforce 
group identity and strengthen connections among gang youth. The paper concludes 
with a discussion of the implications of the Boston gang streetworker program findings 
for gang violence reduction policy and practice. 


Unanticipated outcomes of crime prevention programs that grouped 
delinquent peers 


In one of her final published papers titled “Cures That Harm,” Joan McCord (2003) 
lamented that researchers, practitioners, and policy makers were too narrow in their 
quest to identify effective crime prevention programs. She argued that it was inappro- 
priate to simply ask whether a program “worked or not” as this question failed to 
recognize that well-intentioned treatments can sometimes cause harm. Well-planned 
and adequately-executed crime prevention programs provide no guarantees for public 
safety or crime prevention efficacy. Programs could increase crime, drug use, diminish 
life coping skills, result in premature death, or undermine deterrent impacts of criminal 
justice sanctions. McCord (2003:18) powerfully stated, “Social programs deserve to be 
treated as serious attempts at intervention, with possibly toxic effects, so that a science 
of intervention can prosper.” She supported this assertion by reviewing her experiences 
in detecting harmful outcomes in the follow-up analysis of boys in the landmark 
Cambridge-Somerville Youth Study. 

The Cambridge-Somerville Youth Study began in 1939 as an experiment in which 
650 boys from the Boston area cities of Cambridge and Somerville were matched into 
pairs based on data obtained from physical examinations, interviews with the boys and 
their families, and detailed case histories (McCord 1978; Powers and Witmer 1951). 
Based on a coin flip, boys in each pair were randomly allocated to treatment and control 
conditions. Boys in the treatment group received individual counseling and home 
visiting while boys in the comparison group received no services. The full range of 
treatment services included: regular meetings between the counselors and the boys, 
recreational activities and trips, tutoring in math and reading, encouragement to attend 
church, attendance at summer camps, and the provision of advice and general support 
to the boys’ families (McCord and McCord 1959). Due to austerity measures following 
the United States’ entry to World War II, the program was scaled back to include only 
253 pairs of boys. The program lasted an average of five and a half years when the boys 
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were between 10 and 16 years of age. Initial evaluation results were disappointing, with 
almost equal numbers of treatment and control boys doing better than anticipated at the 
commencement of the program (Powers and Witmer 1951). 

Between 1975 and 1981, McCord and her research team were able to locate roughly 
98 % of the 253 matched pairs of boys. Unfortunately, her follow-up analyses of the 
Cambridge-Somerville Youth Study participants found a variety of adverse program 
impacts. Relative to the control boys, the treated boys were more likely to have been 
arrested for a serious crime, died an average of five years younger, and were more 
likely to have received a medical diagnosis as alcoholic, schizophrenic, or manic 
depressive (McCord 1978, 1981, 1992). She also found that adverse program effects 
increased with increased intensity and duration of treatment, and that adverse effects 
only occurred among boys whose families cooperated with the program. Finally, 
repeated placement in summer camp resulted in strong iatrogenic impacts (Dishion 
et al. 1999). Close association with summer camp peers that the boys wanted to impress 
seemed to lead to increased deviance. As McCord (2003:22) explained: 


At summer camp, misbehaving boys would have unsupervised time during which 
they would be likely to brag about deviance. A bragging effect would be 
particularly noticeable among those sent to camp more than once. After the first 
summer, these boys would have known what camp was like and be in a position 
to estimate the effects of their reported daring (whether or not these reports were 
factual). 


McCord (2003) identified four other rigorous evaluations of well-meaning crime 
prevention programs that yielded unanticipated bad outcomes for participants. These 
programs often involved the grouping of delinquent peers. A randomized experiment 
testing the impacts of a court volunteer program to provide group and individual 
counseling and tutoring services to juvenile probationers found that treated subjects 
committed more crimes relative to control subjects (Berger et al. 1975). Two random- 
ized experiments evaluating group interaction training programs, generally designed to 
improve the social skills of potential delinquents by giving them practice in discussing 
issues with well-adjusted peers, found increases in misbehavior and delinquency in 
high school students (Gottfredson 1987) and problematic behavior among aggressive 
youth (Dishion and Andrews 1995) relative to their untreated counterparts. A random- 
ized experimental evaluation of an activity program designed to provide healthful 
recreation to delinquent adolescents found that treated youth significantly increased 
recidivism after the program stopped relative to control youth (Dufty and Richard 
1978). Finally, “Scared Straight” and other juvenile awareness programs, intended to 
halt offending by directly exposing groups of delinquent youth to stressful prison 
experiences (i.e., during a brief prison visit, adult inmates intimidate the youth in an 
attempt to scare them straight), have been found to increase recidivism among treated 
youth relative to control youth (Lewis 1983; Petrosino et al. 2004). 

Other reviews of individual-based crime prevention program evaluations have also 
identified unexpected harmful effects that seem to be associated with delinquent peer 
grouping (Cécile and Born 2009; Rhule 2005). These reviews further suggest that 
harmful effects vary by program types and the negative influence of delinquent peers 
can be reduced by including more structured activities, enhanced supervision of group 
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interactions, the inclusion of pro-social peers in groups, and familial involvement. 
Weiss et al. (2005), however, suggest that the harmful impacts of delinquent groupings 
in crime prevention programs have been overstated. They argue that the exposure that 
occurs during these programs represents only a fraction of the negative peer influence 
these youth experience on a daily basis. Moreover, Weiss et al. (2005) observed that 
negative peer socializing is not part of the treatment so it is incorrect to suggest that the 
programs are causing the deviant behavior. 

More recently, Welsh and Rocque (2014) conducted a methodologically-informed 
review of 15 Campbell systematic reviews of individual-based crime prevention 
programs that included some 574 studies with 645 independent effect sizes. They 
identified 22 harmful or undesirable program effects from 22 unique studies of 
individual crime prevention programs (3.4 % of 645 effect sizes). Underscoring the 
importance of comprehensive searches for eligible program evaluations in the grey 
literature, roughly two-thirds of these adverse outcomes were reported by unpublished 
studies. Harmful and undesirable impacts were reported in evaluations of a wide 
range of interventions, from anti-bullying programs at schools to second re- 
sponder interventions involving the police. Boot camps and drug courts 
accounted for the largest share of studies with harmful effects. Welsh and 
Rocque (2014) concluded that the grouping of deviant peers, along with implementa- 
tion and theory failure, was a leading explanation for the observed harmful effects in 
these crime prevention programs. 


Theorizing harmful impacts of grouping delinquent peers in prevention 
programs 


The “construct theory of motivation” was developed to understand how close associ- 
ation with peers that youth wanted to impress could lead to increased deviant behavior. 
Construct theory, also known as deviancy training, suggests that people construct their 
motives by the way they perceive choices and their perceptions are shaped by the 
perceived actions of their peers (McCord 1997, 1999, 2000). This theory was applied to 
understand the harmful effects of summer camp on the Cambridge-Somerville Youth 
Study boys (Dishion et al. 1999). Essentially, the summer camp boys took their 
behavioral motivational cues from each other. Delinquent behavior was expect- 
ed to enhance their reputation in the eyes of their peers. As such, they were 
more likely to engage in deviant behavior than boys who did not attend 
summer camp. In this way, antisocial behavior was contagious within groupings 
of delinquent peers. 

A small number of empirical studies have explicitly examined the problem of 
deviancy training in crime prevention programs (e.g., see Gottfredson 2010). For 
instance, in Montréal, an analysis of the impact of varying juvenile court sanctions 
on the later criminal behavior of treated youths found that criminogenic impacts were 
associated with enhanced individual supervision and intensive supervision activities 
that aggregated youth into peer groups (Gatti et al. 2009). In their analysis of an after- 
school program, Rorie et al. (2011) found that juveniles had positive reactions to 
deviant behavior of their delinquent peers and that these positive reactions were more 
likely to occur during less-structured activities. 
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Harmful program effects have also been associated with other causal mechanisms. 
Labeling theory suggests that the terms used to describe and classify individuals may 
determine or influence their behavior (Becker 1973; Lemert 1951; Tannenbaum 1938).! 
Stigmatizing labels are suggested to exert negative influences on a person’s self- 
concept and social identity. Research has supported the perspective that deviant 
individuals exhibit more problematic behavior after being labeled (Lopes et al. 2012; 
Paternoster and Iovanni 1989; Sampson and Laub 1997). For instance, participation in 
juvenile justice interventions has been suggested to confer a stigmatizing label on youth 
that can increase delinquent behavior. Gatti et al. (2009) suggested that negative 
impacts of the juvenile justice program they studied may have been due to both labeling 
effects and deviancy training effects. 

Ekblom and Pease (1995) suggest more general explanations for harmful program 
effects: measurement failure, theory failure, and implementation failure. Measurement 
failure occurs when an evaluation design is not rigorous enough or executed properly to 
detect the “true” impact of the program being tested. Well-designed and implemented 
randomized experiments and quasi-experiments generally address measurement prob- 
lems. Theory failure occurs when the prevention mechanism or idea was unsound from 
the outset. Faulty prevention frameworks, that may have unexpectedly romanticized 
inmate lives and/or challenged youth to prove they were not scared, have been suggested 
to generate the backfire effects observed in scared straight programs (Finckenauer 1982). 
Implementation failure occurs when the prevention idea was sound but not properly 
implemented. Welsh and Rocque (2014) suggest that implementation problems were the 
likely causes of adverse impacts in some of the drug court programs and incarceration- 
based drug treatment programs that they reviewed. The Cambridge-Somerville Youth 
Study is recognized as being a rigorously-designed longitudinal randomized experiment 
with a well-implemented treatment (Zane et al. 2015). As such, the iatrogenic effects 
observed by McCord (2003) seem likely to be generated by theory failure. 

Gang involvement has been demonstrated to have a very large negative effect on 
individual behavior among gang members (e.g., see Thornberry et al. 2003). Gang 
cohesion, often described as group solidarity and characterized by the robustness of 
group identities and the strength of relationship ties among gang members, has long 
been theorized to be related to gang crime and delinquency (Klein 1971; Spergel 1995). 
A stronger sense of “groupness” amplifies internal social processes and potentially 
generates unique gang norms, behaviors, and structures (Papachristos 2013; Thrasher 
1927). Higher levels of cohesion among members of particular gangs are associated 
with increased levels of criminal and delinquent behavior. A solid theoretical and 
empirical case can be made that gang intervention programs that emphasize group 
processes already in place in gangs can increase gang cohesiveness and, therefore, gang 
crime (Klein and Maxson 2006). Gang outreach worker programs, commonly called 


' Beyond the application of construct theory to understand the strong backfire effects of summer camp, 
McCord (1981) developed other hypotheses to explain the harmful effects of the Cambridge-Somerville Youth 
Study on treated boys relative to untreated boys. Most notably, McCord (1981) suggested that the Cambridge- 
Somerville Youth Study boys in the treatment group may have suffered a labeling effect. Other possible 
explanations included (1) the counselors imposed middle-class values on lower-class youth that simply did not 
work for them, (2) the boys in the treatment group became dependent on the counselors and lost a key source 
of support when the program ended, and (3) the support of the counselors raised the treatment boys’ 
expectations that could not be sustained, and disillusionment set in after the program was completed. 
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“streetworker” programs, have been found to strengthen group identity, increase gang 
cohesion, and, as a result, generate higher levels of antisocial behavior among 
gang members (Klein 2011). These unintended harmful effects have been found 
in classic studies of gang streetworker programs (Klein 1971) and modern 
program evaluations of streetworker programs designed to reduce serious gang 
violence (Wilson and Chermak 2011). 


Adverse program effects in early street worker programs 


Four high-quality evaluations inform our understanding of adverse program effects in 
early (pre-1970) streetworker programs.” During this time period, delinquency reduc- 
tion programming was generally designed to “transform” delinquent gangs into pro- 
social groups (Spergel 2007). The first rigorous evaluation tested the “Total 
Community Gang Control Project” operating in Boston’s Roxbury neighborhood 
between 1954 and 1957 (Miller 1962). This project was designed to reduce neighbor- 
hood adolescent delinquency by deploying an intervention aimed at three ecological 
levels: the community, the family, and the gang. The main delinquency reduction 
intervention involved the assignment of seven professionally-trained streetworkers to 
develop relationships with and deliver services to 21 Roxbury gangs. Seven of these 
targeted gangs received “intensive” attention from at least one of the streetworkers. 
Streetworkers were directed “to contact, establish relations with, and attempt to change 
resident gangs” (Miller 1962: 169). Unfortunately, the quasi-experimental impact 
evaluation revealed that the treatment gangs showed no improvement on a variety of 
delinquency measures when compared to counterpart control gangs. Rather, increases 
in delinquency were detected among several categories of offending—particularly 
increases in serious offending among younger gang members, and among males 
relative to their female counterparts (Miller 1962). 

The Chicago Youth Project, an initiative of the Chicago Boys’ Clubs from 1960 through 
1966, followed the same basic logic of Miller’s “total community” approach. However, 
this project placed greater emphasis on community organization, included outreach to non- 
gang youth, and made greater use of data and research to refine its outreach efforts (Gold 
and Mattick 1974). An evaluation led by researchers at the University of Michigan’s 
Institute for Social Research revealed that the targeted gang-based streetworker interven- 
tions did not generate any delinquency prevention gains. Moreover, the greatest delin- 
quency increases were experienced by youth who were served most intensively by their 
assigned streetworkers. While the project showed some promise in finding employment 
opportunities and reengaging school dropouts, the quasi-experimental evaluation results 
suggested that youth living in the target intervention neighborhoods were slightly worse off 
on a variety of delinquency and pro-social indicators than youth living in control neigh- 
borhoods (Gold and Mattick 1974; Spergel 1995). 


? The well-known Chicago Area Project (CAP), designed by University of Chicago sociologist Clifford Shaw, 
was one of the earliest and most influential gang streetworker programs. The CAP model employed local adults 
to outreach gang youth with group activities and social service opportunities in order to drive down 
neighborhood crime. Spergel (2007) has noted that CAP’s outreach workers were among the first “curbstone 
counselors”—individuals with ties to both the neighborhood and gangs dedicated toward group and individual 
transformation. Unfortunately, CAP was never rigorously evaluated to establish its impacts on gang delinquency. 
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The Los Angeles Group Guidance Project was sponsored by the Los Angeles 
County Probation Department and operated between 1961 and 1965 (Klein 
1969, 1971). The Group Guidance Project employed a “transformational” 
streetworker approach to engage four majority-black Los Angeles gangs, broken 
into 16 subgroups comprised of approximately 800 members. This project relied 
heavily on a streetworker-led group programming approach which featured 
surprising little employment, educational, and community organizing content. 
Much of the group-based programming involved meetings, sports, trips, and 
other recreational activities. Similar to previous streetworker evaluations, Klein 
(1969, 1971) found that the Group Guidance Project was associated with a 
significant increase in delinquency among the gang members served, and the 
gangs most intensively served performed the worst on a variety of delinquency 
indicators. Drawing on carefully constructed streetworker contact data and 
ethnographic field notes, Klein (1969, 1971) reasoned that gang delinquency 
increased as a result of enhanced gang cohesion and increased gang recruitment 
fueled by large amounts of group-based programming. 

Klein (1971) specifically designed the 1965-1966 “Ladino Hills Project” to 
avoid the adverse program impacts of the Group Guidance Project. This Los 
Angeles-based project engaged a single Mexican gang for 18 months, with a 
six-month follow-up period of data collection. The Ladino Hills project dis- 
pensed with all group programming in an effort to reduce gang cohesion and, 
as a result, gang delinquency. In place of the group-based programming 
employed by streetworkers in the Group Guidance Project, the Ladino Hills 
streetworkers (aided by Klein’s research staff) provided educational and 
employment opportunities to gang youth that were explicitly designed to 
wean away members from the gang. While the project did not significantly 
affect rates of offending for gang members, Klein (1971) reported that overall 
crime by the gang was reduced by curtailing gang membership and recruitment. 


Investigating potentially harmful impacts in contemporary gang 
streetworker programs 


Streetworker programs implemented in the 1950s and 1960s generally sought to 
control gang fighting, address individual problems through social services, 
provide access to opportunities, change delinquent values, and reduce individual 
and group delinquency (e.g., see Spergel 1966). Contemporary streetworker 
programs share these goals and have placed a strong emphasis on reducing 
community violence (Hureau et al. 2014). These early streetworker programs 
failed to reduce gang delinquency and, as described above, some programs 
increased delinquent behavior. Although a variety of streetworker and gang 
outreach programs proliferated over the course of the youth violence epidemic 
of the late 1980s and early 1990s (e.g., see Kennedy 2011), most of these 
programs were not formally evaluated. Evaluations of contemporary streetworker 
programs have generally found mixed impacts on gang violence (Papachristos 2011; 
Skogan et al. 2009; Webster et al. 2012), with several studies showing null and harmful 
effects (Fox et al. 2015; Wilson and Chermak 2011). Table 1 provides a summary 
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Table 1 Summary comparison of classic and contemporary streetworker program evaluations 


“Classical” “Contemporary” 
Outcome Delinquency Gun violence 
Intervention theory Gang transformation Public health/violence interruption 
Implementation Full; aided by researchers Mixed; implementation failures 
Key findings Null or increased delinquency Mixed; proclaimed “successes” 
Unit of analysis Group/individual Areas/neighborhoods 


Adapted from Hureau et al. (2014) 


comparison of key aspects of gang streetworker program evaluations from the 
1970s and earlier (denoted as “classic” studies) with more contemporary gang 
street worker evaluations. 

Recent interest in the evaluation of streetworker programs has been driven by 
CeaseFire Chicago (now known as Cure Violence; http://cureviolence.org/) and the 
various replication demonstrations it has inspired. In the CeaseFire Chicago gang 
violence reduction model, one set of outreach workers maintains contact with at-risk 
individuals and aims to broker services and pro-social opportunities, while “violence 
interrupters” focus solely on the mediation of violent disputes and the prevention of 
retaliation. Other facets of CeaseFire’s programming extend beyond the streetworker 
approach to reveal its public health underpinnings. Specifically, the program aims to 
promote broad-based, population-level shifts in attitudes toward the acceptability of the 
use of violence, akin to previous public health campaigns targeting issues such as 
smoking cessation and seatbelt use. Furthermore, the CeaseFire model prescribes the 
promotion of various community-level campaigns aimed at both community attitude 
change and the enrollment and mobilization of community members in responding to 
high-profile violent events (Papachristos 2011; Skogan et al. 2009; Wilson and 
Chermak 2011). 

The available evidence on the violence reduction impacts of this influential 
streetworker program is mixed, with more evidence supporting positive program 
effects. Skogan et al. (2009) used a quasi-experimental design to compare 
shooting trends in neighborhoods that experienced the CeaseFire Chicago inter- 
vention relative to shooting trends in comparison neighborhoods. Their analyses 
suggested that CeaseFire neighborhoods experienced significantly fewer shoot- 
ings (ranging from 16 % to 34 %) when compared to their comparison 
counterparts. However, companion analyses of the CeaseFire Chicago program, 
using slightly different datasets and other analytic approaches, found that the 
program generated much more modest impacts, with roughly half of the treated 
neighborhoods demonstrating significant violence reductions while the others 
showed a null effect (see technical appendices by Block and Papachristos in 
Skogan et al. 2009; Papachristos 2011). Using a similar longitudinal quasi- 
experimental design, Henry et al. (2014) recently found a revamped CeaseFire 
Chicago streetworker intervention, which included strategic partnerships with 
the Chicago Police Department, was associated with significant reductions in 
violence in treated neighborhoods relative to comparison neighborhoods. 


DQ Springer 


10 A.A. Braga 


Subsequent replication streetworker programs have generated much more mixed 
results.> A quasi-experimental evaluation of Newark’s Operation Ceasefire program, 
which blended Chicago CeaseFire streetworker violence interruption tactics with 
strategic law enforcement action, found no evidence of a statistically significant 
decrease in gunshot injuries in the Ceasefire areas compared with similar neighbor- 
hoods (Boyle et al. 2010). Pittsburgh’s “One Vision One Life” program used 
streetworkers to reduce gang violence through conflict mediation and the provision 
of services and opportunities to at-risk individuals. Using a quasi-experimental, neigh- 
borhood-level, difference-in-differences analysis, program evaluators found that the 
One Vision program had no effect on the incidence of homicide and was associated 
with a statistically significant increase in aggravated assaults and gun assaults in its 
target neighborhoods (Wilson and Chermak 2011). The evaluators specifically noted 
that the presence of streetworkers may have increased the cohesion of gangs, making 
some groups more organized, in turn leading to increased violence. 

Baltimore’s “Safe Streets” streetworker program represents a rigorous Chicago 
CeaseFire replication that was piloted in four of Baltimore’s most violent neighbor- 
hoods between 2007 and 2010. Webster and colleagues (2012, 2013) used a quasi- 
experimental design to analyze Safe Streets’ impact on homicide and non-fatal gun 
violence trends in treated areas relative to homicide and non-fatal gun violence trends in 
comparison areas. Importantly, the evaluation attempted to control for other law 
enforcement and social service interventions affecting the incidence of gun violence. 
Only one program site (Cherry Hill) evidenced significant declines in both homicides 
and nonfatal shootings, although two others showed overall reductions in gun violence 
(driven largely by decreases in nonfatal shootings). A single program site (Madison- 
Eastend) experienced a statistically significant increase in homicides, accompanied by 
a significant decrease in nonfatal shootings. The Phoenix TRUCE Project was also 
modeled after the Chicago CeaseFire program. While community partnerships were 
weak, program evaluators found that this gang streetworker program engaged in a 
strong media campaign, conducted conflict mediations, and identified high-risk indi- 
viduals for case management (Fox et al. 2015). A quasi-experimental evaluation found 
that the TRUCE program generated reductions in overall violence but also resulted in 
significant increases in shooting incidents. 

One key limitation of the available scientific evidence on the impact of these 
contemporary streetworker programs on serious violence is that changes in group and 
individual violent behaviors are not directly measured by program evaluators (Hureau 
et al. 2014). Unlike the “classic” streetworker evaluations (e.g., Klein 1971; Miller 
1962), contemporary evaluations analyze aggregated area-level data on violent out- 
comes in treated and untreated areas. Even with comparison areas, it remains difficult to 
know whether the behavior of treated groups and the individuals that comprised them 
significantly changed as a result of the intervention without directly measuring out- 
comes at more refined units of analysis. With the support of colleagues, the author of 


3 The scientific evidence reviewed here excludes the evaluation of the “Save Our Streets” (SOS) streetworker 
program in the Crown Heights neighborhood of Brooklyn (Picard-Fritsche and Cerniglia 2013) due to a very 
weak evaluation design that did not adequately control for rival causal factors. The evaluation also made 
implausibly large violence reduction claims for a police district of some 96,000 residents given the scale of the 
program (only four outreach workers who managed just 96 clients—only 68 % of whom were classified as 
“high risk” for involvement in gun violence). 
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this article served as the Principal Investigator of a gang streetworker program evalu- 
ation enterprise explicitly designed to determine whether the intervention produced any 
significant changes in violent group behavior in Boston, Massachusetts (Hureau et al. 
2014). This unpublished work is summarized here. 


The StreetSafe Boston gang streetworker program 


Like many cities, serious gun violence in Boston is concentrated among a small number 
of gangs engaged in ongoing conflicts (Braga 2003; Kennedy et al. 1996). A recent 
analysis found that only 1 % of Boston’s youth between the ages of 14 and 24 were 
gang members, but that Boston gangs generated more than half of gun homicides and 
were involved in more than two-thirds of non-fatal shootings as victims, offenders, or 
both (Braga et al. 2008). Launched by The Boston Foundation (TBF) at the end of June 
2009, StreetSafe Boston (SSB) was a gang streetworker program specifically aimed at 
reducing serious gun violence by gangs in the South End, Dudley, Grove Hall, 
Bowdoin-Geneva, and Morton-Norfolk neighborhoods of Boston (Hureau et al. 
2014; http://www.tbf.org/tbf/61/ssb). The initiative was framed as an action research 
project and Harvard University researchers were engaged to provide ongoing analytic 
support to guide implementation and conduct a rigorous program evaluation. When SSB 
was launched, the Harvard research team identified 66 Boston gangs that were actively 
engaged in gun violence. In consultation with the Boston Police Department (BPD) and 
other partner agencies, TBF subjectively selected 20 gangs for the SSB treatment based on 
their association with one of the five targeted neighborhoods, historic and current levels of 
gun violence, and perceived risk for continued violent gun behaviors. 

Many readers will question the decision made by TBF to implement a gang 
streetworker program given the existing evaluation literature suggesting harmful effects 
associated with these kinds of programs. It is important to note here that the former 
Vice President of Programs for TBF and lead architect of the SSB program, Robert 
Lewis Jr., was once a city-employed streetworker in Boston during the 1990s. Drawing 
on his personal experiences and strong claims of noteworthy gang violence reduction 
gains being made by Chicago CeaseFire, Lewis was convinced that gang streetworker 
programs could be used to good effect in reducing gun violence in Boston. As such, 
Lewis played a central role in designing, raising operational funds, and implementing 
the SSB program. 

The SSB intervention was comprised of two complementary violence reduction 
strategies (Hureau et al. 2014). The first strategy involved street-level conflict negotiation 
and mediation work conducted by streetworkers. Briefly, the SSB streetworker strategy 
depended on the ability of streetworkers to connect and build meaningful relationships 
with the gang members they served in order to mediate and resolve gang conflicts and/or 
interrupt cycles of ongoing gang violence. SSB generally deployed 15 streetworkers to 
develop relationships with and serve members of the 20 treatment gangs. Most 
streetworkers were assigned to a single gang and a small number of streetworkers were 
assigned to serve two closely-allied gangs. While a single streetworker served as the 
primary point of contact for each gang, streetworkers often collaborated in serving 
treatment gangs. This was especially true of streetworkers working in the same neigh- 
borhood and during times of stress and conflict when strategic problem-solving ap- 
proaches were required to mediate disputes and negotiate resolutions. 
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The second SSB violence reduction strategy involved the delivery of social services 
to treatment gang members. Complementing the work of the streetworkers, program 
coordinators addressed the service needs of individuals embedded in treatment gangs 
by offering case management and needs assessment services, serving as brokers to 
social service and opportunity provision programming in Boston, and steering individ- 
uals away from gang-involved activities and toward structured life-skills, education, 
and employment opportunities. In addition to referrals to external service providers, the 
“in-house” StreetSafe Transitional Employment Program (STEP) blended three weeks 
of intensive pre-employment training with three months of subsidized employment 
through community employment partners. 

The Harvard research team conducted process and impact evaluations (Hureau et al. 
2014). The methods and findings of the impact evaluation are summarized in the next 
section. The process evaluation involved a blend of quantitative analyses of adminis- 
trative data, ethnographic observations of work performed by program coordinators and 
streetworkers, and extensive interviews of SSB staff, partnering agencies, and treated 
gang members. While SSB experienced some implementation challenges, the process 
evaluation concluded that the program was well managed and achieved full implemen- 
tation (Hureau et al. 2014). In particular, the process evaluation noted that SSB staff 
maintained deep and broad relationships with treatment gang members and, by its 
fourth year of implementation, streetworkers had engaged more than 92 % of the 426 
individuals known to be active participants in the treatment gangs. 


Impact evaluation 


A non-randomized, quasi-experimental design was used to analyze serious gun vio- 
lence trends for Boston gangs that received the SSB treatment relative to serious gun 
violence trends for comparison Boston gangs that did not receive the SSB treatment 
(Shadish et al. 2002).* Serious gun violence was measured using BPD computerized 
records of assault and battery with firearm incident reports (hereafter “shooting” 
incidents). These data represent shooting events where guns were fired and victims 
were physically wounded by the fired bullets. The “crime incident review” process was 
used to determine the motive for the shooting incidents, whether a shooting event 
involved a gang member as a suspect, victim, or both, and the specific gangs involved 
(Braga et al. 2014b; Klofas and Hipple 2006). The units of analysis in the SSB 
evaluation were quarterly counts of shootings by and against specific Boston gangs 
between July 1, 2006 and June 30, 2013. 

Since gangs were not selected randomly for treatment, the Harvard research team 
needed to develop a comparable counterfactual group of gangs to determine whether 
the SSB intervention generated the desired gun violence reduction impact. Drawing on 
prior evaluation designs used to detect changes in the gun violence behaviors of Boston 


* Using the Maryland Scientific Methods Scale (Sherman et al. 1997) as a standard, the SSB quasi- 
experimental design with matched treatment and control groups would be considered a “Level 4” evaluation 
as it measured outcomes before and after the program in multiple treatment and control condition units. These 
types of designs have better statistical control of extraneous influences on outcomes and, relative to lower- 
level evaluations, deal with selection and regression threats more adequately. The Scientific Methods Scale 
ranked scientific studies from Level | (weakest) to Level 5 (strongest) on overall internal validity. Properly 
implemented randomized experiments were rated highest on the scale and observational studies lowest. 
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gangs subjected to focused deterrence programs (Braga et al. 2013, 2014a), comparison 
gangs were identified by analyzing basic demographic information and covariates 
associated with violent gang behavior. These covariates included pre-treatment shoot- 
ing counts, gang size, gang race/ethnicity, number of active conflicts with rival gangs, 
gang longevity, gang turf located in a housing project area, and concentrated disadvan- 
tage in the neighborhood surrounding gang turf. Three different analytic approaches 
were used to match treated SSB gangs to comparable untreated Boston gangs: 
theoretically-informed matching (Rossi et al. 2006), propensity score matching 
(Rosenbaum and Rubin 1983, 1985) and coarsened exact matching (Blackwell et al. 
2009; Iacus et al. 2012). With each matching method, comparison groups were 
selected from the available pool of 46 potential comparison groups. Each 
potential comparison group was a non-SSB gang active in Boston during the 
entire 7-year evaluation period. 

The three distinct matching methods produced similar results, bolstering the evalu- 
ation team’s confidence in the overall findings. The results from the theoretically- 
informed matching exercise are described here; interested readers should review 
Hureau et al. (2014) for the full set of results. A variation of a multi-level negative 
binomial regression model was used to analyze the quarterly change in gang-involved 
shootings for treatment and comparison gangs over a 7-year observation period 
(n=28 quarters). Specifically, growth curve regression models (Gelman 2005; 
Singer and Willet 2003) were used to analyze panel data on street gang 
shooting incidents over the evaluation time period. A difference-in-differences 
(DID) estimator (see, e.g. Card and Krueger 1994) was used to estimate of the 
impact of the SSB treatment on total shootings involving treated gangs (n=20) 
relative to total shootings involving comparison gangs in Boston (n=20). The 
growth curve regression models also included covariates to control for seasonal 
variations and linear and non-linear secular trends. 


Results 


From the perspective of a one-group-only pre—post comparison, the SSB treatment 
seemed to prevent gun violence by gangs subjected to the intervention. During the three 
years preceding the launch of the program, the 20 SSB gangs averaged 112.3 total 
shootings per year. After the SSB program was implemented, the treated gangs 
averaged 86.8 total shootings per year during the 4-year treatment period, suggesting 
a 23 % reduction in gang-involved shootings committed by and against the treated 
gangs. Unfortunately, the quasi-experimental evaluation revealed the 20 matched 
untreated gangs experienced a steeper reduction in total shootings during this same 
time period. Figure | shows the trend in the mean yearly number of total shootings for 
the 20 SSB gangs as compared to the trend in the mean yearly number of total 
shootings for the 20 matched untreated gangs. SSB gangs averaged 5.6 total shootings 
per year during the pre-intervention period and averaged 4.3 total shootings per year 
during the treatment time period, representing a 23 % reduction in total shootings. Total 
yearly mean shootings by matched untreated gangs, however, decreased by 31 % from 
4.9 shootings per year during the pre-intervention period to 3.4 shootings per year 
during the treatment time period. 


DQ Springer 


14 A.A. Braga 


Mean SSB and Comparison Group Shootings by Year 
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Fig. 1 Adapted from Hureau et al. (2014) 


The growth curve regression models confirmed that the SSB gangs experienced 
higher levels of gun violence when compared to the matched untreated gangs. 
Controlling for trends and seasonal variations, the DID estimator found that the SSB 
treatment was associated with a statistically significant 14 % increase in total shootings 
when comparing pre-intervention and intervention periods for the treatment and com- 
parison gangs (p<.05). The statistical analysis further suggested that the observed 
“backfire effect” was influenced by a larger increase in shootings committed by 
treatment gangs. For incidents where gangs were identified as the suspected shooters, 
the SSB treatment was associated with a statistically significant 17 % increase in 
shootings committed by treated gangs relative to shootings committed by matched 
untreated gangs (p<.05). For incidents where gangs were identified as shooting victims, 
the SSB treatment was associated with an estimated 10 % increase in shootings suffered 
by treated gangs relative to shootings inflicted upon matched untreated gangs (p<.10). 

It is worth noting here that the SSB evaluation would have revealed a noteworthy 
reduction in total shootings by and against treated gangs if the research design did not 
include comparison gangs. Similarly, McCord (2003) noted that, without a control group, 
evaluators would have concluded that the Cambridge-Somerville Youth Study was effec- 
tive in reducing delinquent behavior. This underscores the importance of rigorous evalu- 
ation designs that include counterfactuals in detecting potentially harmful program impacts. 


Discussion 


It seems very unlikely that the iatrogenic impacts observed in the SSB evaluation were 
due to implementation failure. Indeed, the process evaluation concluded that the SSB 
program was well managed and, over the course of four years, streetworkers managed 
to keep ongoing contact with the vast majority of the members of the treated gangs. 
Hureau et al. (2014) further reported that streetworkers engaged in conflict negotiation 
and mediation work and that many gang members received services and opportunities 
through the program’s service delivery strategy. The SSB program seemed to be 
implemented as planned. 
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The SSB evaluation also did not suffer from measurement failure. The SSB quasi- 
experimental design used matched treatment and control groups and measured out- 
comes before and after the program in multiple treatment and control condition units. 
The SSB evaluation also directly measured whether treatment and control gangs 
changed their violent gun behaviors during the study time period. In a series of interim 
reports (Hureau et al. 2014), the Harvard research team conducted supplementary area- 
level quasi-experimental analyses to determine whether the program had broader 
impacts on treated neighborhoods relative to untreated areas of Boston. These analyses 
yielded null program effects. If the SSB evaluation was limited to area-level analyses, 
like other contemporary evaluations of streetworker programs (Boyle et al. 2010; 
Skogan et al. 2009; Webster et al. 2012, 2013), the adverse effects of the program on 
gang behavior would not have been detected. 

The research team consistently reported in interim reports to the SSB program staff 
that comparison gangs were involved in lower levels of gun violence when compared 
to gun violence generated by the treatment gangs. However, the outcome differences 
between the two groups were not statistically significant during Years 2 and 3. 
The evaluation team noted the concerning trends but, during that time period, 
reported the differences as null effects to TBF overseers and SSB program 
staff. Since treated gangs were experiencing decreasing levels of gun violence 
during the post-implementation time period (Figure 1), these findings were 
generally interpreted by SSB staff as evidence that the treatment gangs needed 
stronger dosages of SSB violence prevention work rather than evidence of 
potentially harmful impacts. The harmful impact of the SSB intervention was 
not noted as a statistically significant effect until the final evaluation report was 
completed at the end of Year 4. 

It seems likely that the harmful impacts of the SSB streetworker program were 
generated by theory failure. The SSB program was powerfully influenced by the 
apparent success of CeaseFire Chicago and program developers drew directly on key 
CeaseFire intervention ideas. Little attention seemed to be paid to the extended 
criminological and sociological literature on the potential backfire effects that may 
occur when doing prevention work with delinquent groups more generally and using 
streetworkers to change violent behaviors of gangs specifically. While the Harvard 
research team is in the process of empirically investigating SSB program dynamics, 
anecdotal evidence from the process evaluation suggests that the SSB intervention may 
have strengthened gang identity and cohesion by its extensive use of group-based 
programming (Hureau et al. 2014). Indeed, given the harmful effects demonstrated in 
prevention programs that treat groups of delinquents (McCord 2003; Welsh and 
Rocque 2014), classic gang streetworker programs (Klein 1971; Miller 1962), and 
some contemporary gang streetworker programs (Wilson and Chermak 2011), the 
adverse effects observed in the SSB evaluation are not surprising. 

Moreover, SSB exclusively hired former gang members, many of whom had spent 
extensive time in prison, as streetworkers. In his critical analysis of contemporary 
streetworker programs, Klein (2011) identified pressing issues that need to be consid- 
ered when using former gang members as streetworkers. Given their “street cred” and 
life experiences, former gang members are well positioned to establish rapport with 
current gang members and usually have a deep knowledge of local gang conflicts and 
alliances. However, when sharing past experiences, former gang members can 
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inadvertently glorify gang life while developing relationships with active gang mem- 
bers and foster gang identification and cohesion by reifying group processes and 
dynamics. Moreover, streetworkers who are former gang members have been reported 
to hinder interagency collaborations due to their reluctance to share information on their 
assigned gangs with partnering agencies (especially law enforcement agencies). Finally, 
former gang members sometimes return to criminal behavior while serving as gang 
interventionists; as Klein (2011:1039) observed, ““You can take the member out of the 
gang, but sometimes you cannot take the gang out of the member.” Over the course of 
the program, four SSB streetworkers were arrested for charges ranging from assaulting 
a police officer to counterfeiting to drug dealing (Allan 2014). All four were subse- 
quently fired by TBF. 

In late fall 2014, TBF made the decision to close down the SSB streetworker 
program due to budget constraints and the negative program evaluation results. The 
City of Boston, however, decided to hire the SSB streetworkers to supplement the 
Boston Centers for Youth Families (BCYF) streetworker program (Allan 2014). BCYF 
streetworkers engage in similar “violence interruption” work but are more generally 
assigned to work in specific neighborhoods and connect both gang and non-gang youth 
to services and opportunities. Drawing on the lessons from the SSB experience, the 
BCYF streetworker program, and gang streetworker programs operating in other 
jurisdictions, would be well advised to focus their gang outreach practices on under- 
standing and responding to group dynamics without reinforcing group identity and 
cohesion. Streetworkers can be oriented towards addressing individual situations rather 
than focusing on bringing together groups of gang members. These activities could take 
many forms. For instance, streetworkers should actively attempt to prevent younger 
kids from joining gangs when they first start “hanging around” with older gang peers. 
As previous researchers have noted, overall gang delinquency might be reduced not 
only by addressing group dynamics but also by reducing the size of the gang itself 
(Klein 1971). To this end, streetworkers could prioritize attempts to facilitate peaceful 
exits from gang life and be trained to recognize opportunities for “peeling off” gang 
members whose situations allow for it. 

BCYF streetworkers are also key partners in the Operation Ceasefire focused 
deterrence strategy, a program that has been rigorously evaluated and shown to produce 
significant reductions in gang-related gun violence in Boston (Braga et al. 2001, 2013, 
2014a). Streetworkers add a much needed social service and opportunity provision 
component to law enforcement action in these strategies and help deliver credible anti- 
violence messages to gang members. Indeed, as suggested by Kennedy (2011), 
streetworkers may be effective when they work in partnership with others and as an 
element in a more robust violence prevention strategy. 

The SSB experience, in conjunction with past research on effective gang violence 
prevention in Boston and elsewhere (Braga and Weisburd 2012), suggests that the 
problem of preventing the next gang shooting from happening may be too big for any 
one agency or program to handle by itself. The presence of street gangs, and the 
violence generated by their conflicts, stem from very complicated and longstanding 
underlying social conditions in urban neighborhoods (Bursik and Grasmick 1993; 
Klein 1995). The value of streetworker programs may be better understood as a 
necessary component of a larger network of capacity (Moore 2002) to reduce gang 
violence rather than the sole response to this very complicated problem. 
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Conclusion 


Joan McCord (2003) called for more targeted research to discover whether crime 
prevention programs generated unintended harmful impacts on participants. 
Moreover, she suggested that program evaluators and policy makers needed to pay 
close attention to the complex interplay of program features, client characteristics, 
settings, evaluation design quality, and other salient factors and dynamics when 
considering positive, null, and negative effects. The bulk of the research evidence 
presented here, highlighted by the Boston experience, suggests that gang streetworker 
programs are at high risk of generating unintended adverse outcomes for treated gang 
members relative to their untreated counterparts. Indeed, given the existing evidence 
base, jurisdictions suffering from gang violence problems should be advised to avoid 
implementing gang streetworker programs. Steering policy makers and practitioners 
away from prevention programs that risk generating toxic backfire effects would be 
most consistent with Joan McCord’s legacy. 

Violence prevention programs are unfortunately sometimes implemented without 
full consideration of the extant research evidence. Indeed, the implementation of the 
SSB gang streetworker program was strongly influenced by the personal experiences of 
a high-ranking TBF executive and supported by a single highly positive program 
evaluation of the well-known Chicago CeaseFire streetworker program. In these 
instances, existing and planned programs should be monitored with a high degree of 
vigilance and evaluated with controlled evaluation designs. This will allow harmful 
streetworker programs to be immediately shut down if harmful impacts are observed 
and facilitate the acquisition of new knowledge on program features associated with the 
varying effects shown in evaluations of contemporary streetworker programs. Doing so 
would also be a very appropriate way to honor Joan McCord’s legacy. 
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